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The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)13 Responsive to communication(s) filed on 8-18-2004 . 
2a)^ This action is FINAL. 2b)n This action is non-final. 

3) n Since this application is in condition for allowance except forfomia! matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1 935 CD. 1 1 , 453 O.G. 21 3. 
Disposition of Claims 

4) 13 Claim(s) 1-12,14-19 and 21-24 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 0 Claim(s) is/are allowed. 

6) 13 Claim(s) 1-12, 14-19 and 21-24 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)13 The drawing(s) filed on 27 June 2000 is/are: a)n accepted or b)M objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 !)□ The proposed drawing correction filed on is: a)n approved b)^ disapproved by the Exanniner 

If approved, corrected drawings are required in reply to this Office action. 

12) 0 The oath or declaration is objected to by the Examiner. 

Priority under 35 U.S.C. §§119 and 120 

13) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)nAII b)n Some*c)n None of: ^ 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)), 
* See the attached detailed Office action for a list of the certified copies not received. 

14) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) 0 Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1) □ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). 



2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other; 
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DETAILED ACTION 

Claims 1-12, 14-19 and 21-24 are presented for examination. 

Drawings 

This application has been filed with informal drawings which are acceptable for 
examination purposes only. Formal drawings will be required when the application is 
allowed. 

The amendment filed on 8-18-2004 is entered. 

Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

Claims 1-12, 14-19 and 21-24 are rejected under 35 U.S.C. 112, first paragraph, 
as failing to comply with the written description requirement. The claim(s) contains 
subject matter which was not described in the specification in such a way as to 
reasonably convey to one skilled in the relevant art that the inventor(s), at the time the 
application was filed, had possession of the claimed invention. In independent claims 1 , 
4, 8, 9, 10, 14, 17 and 21, the language "invisible keyboard made in a touch-sensor free 
environment" was never described or mentioned in the specification. Correction is 
required. 
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Claim Rejections - 35 (JSC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-12, 14-19 and 21-24 are rejected under 35 U. S. C. 103 (a) as being 
unpatentable over U.S. Patent # 5,767,842 ("Korth") in view of U.S. Patent # 6,407,679 
("Evans et a!."). 

As for claims 1, 8-10, 14, 17, and 21-24, Korth teaches a camera Fig.1 (2) 
corresponding to one or several cameras, a computer side of the interface there need 
only be a way to detect the fingertip motions of the keyboard operator (column 4, lines 
27-29) corresponding to one or more memories with CPU connected to the cameras, 
and processes running in the CPU that associates gesture movements with typing and 
window may appear only during the input of data (column 4, lines 54-55) corresponding 
to the produce gesture associated textual output, A virtual keyboard 3 serves as data or 
command input device. From the computer side of the interface there need only be a 
way to detect the fingertip motions of the keyboard operator (column 4, lines 25-29) 
corresponding to sensing the typing gestures. It is inherent in the art that the gestures 
are associated with the most probable keys that would be typed if the keyboard were 
presented. Accordingly, Korth teaches all the claimed limitations as recited in claims 1 , 
8-10, 14, 17, and 21 with the exception of providing a classify groups of gestures 
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associate each of the classes with one or more possible keys, assign a probability to 
each of said possible keys, and integrate the probabilities assigned to the possible keys 
to identify a word for a sequence of gestures. 

However, Evans et al. teach (Fig. 1), where each key, or finger, would represent 
several different alphanumeric characters (column 2, lines 40 and 41) and a glove 
usable in a virtual environment where each finger has a group of letters associate with 
one of the keys in the virtual keyboard (column 8, lines 13-30) corresponding to classify 
each gesture into one of the plurality classes depending on the movement of the 
gestures and associate each of the classes with one or more possible keys, assign a 
probability to each of said possible keys, and integrate the probabilities assigned to the 
possible keys to identify a word for a sequence of gestures. It means that each key 
represents a classification of several alphanumeric characters. 

It would have been obvious to utilize the finger labeled as taught by Evans et al. 
in the user command system disclosed by Korth because this would allow the user 
command to match each key sequence with one or more words in the dictionary. 

As for claims 2, 1 1 , and 18-19, Korth teaches a keyboard that should provide the 
user with the key information and feedback. Tactile feedback of virtual keyboard is given 
by the contact of the finger-tip with the working plate (column 7, lines 14-17) 
corresponding to a feedback is provided to the user on what kind of keys are associated 
with the user's gestures. 

As for claims 3 and 12, Korth teaches a visualization of the actual key function 
allows the handling of keyboards with multiple shift keys and subgroup display for large 
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characters sets (column 3, lines 1-4) corresponding to displaying keys on the display, a 
keyboard for musical instruments (column 3, line 1 1 ) corresponding to playing sound 
labels for keys and playing special sound indicators, a video sensor that is used for 
monitoring the keyboard to generate image data representing positions of an operator's 
hand with respect with the keys (column 3, lines 55-58) corresponding to displaying 
image indicators on the display, a camera Fig. 1 (2) corresponding to projecting the 
keyboard to any surface and displaying picture of the keyboard with the user's hand. 

As for claim 4, Korth teaches FIG. 2 shows the area viewed by camera 2. In the 
observed region 4 the user's hands are detected. The enlarged section 5 of FIG. 2 
represents an area of the camera's image which is chosen for further image processing 
corresponding to one or more cameras, another image evaluation procedure then helps 
the image processor to match the virtual keyboard with the tennplate. The place-mat 
may be applied on a piece of soft tissue (e.g. polyurethane foam) to reduce the strain on 
the finger-tips from the impact on a rigid desk plate (column 7, lines 47-52) 
corresponding to one or more memories with CPU connected to the cameras, and A set 
of simple intuitive 'gestures' may be defined to control the operation of a virtual 
keyboard (column 6, lines 7 and 8) corresponding to processes running in the CPU that 
associates gesture movements with typing and produce gesture associated with textual 
output and a gesture capturing module that captures gestures through camera sensors, 
and the shape of the hands can be used to send gestures with additional information to 
the optical interface (column 2, lines 55 and 56) corresponding to an integrator module 
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that integrate sequence of candidates of most probable keys into unique output key 
sequence. 

As for claim 5 and 6, Korth teaches a method of user command or data 
acquisition in a processing system that allows for analysis of human hand and finger 
motions. The activities are then interpreted as operations on a physically non-existent 
input device. The resulting optical user/computer interface complements the visual 
screen-to-user path with a hand-to-image data acquisition link from the user to the 
processor unit. It allows for the emulation of different types of manually operated input 
devices (mouse, keyboard, etc.). Mechanical input units can be replaced by virtual 
devices, optimized for the current application and for the user's physiology. The speed, 
simplicity and avoidance of ambiguity of manual data input will be maintained. Sensing 
of finger motions can generate feedback to reduce the hazard of repetitive strain injury 
(RSI) syndrome (abstract). 

As for claim 7, Korth teaches a method of data input using video sensors for 
monitoring positions of an operator's hand with respect to keys on a virtual keyboard 
optically produced on a surface. 

As for claims 15 and 22, Korth teaches a set of simple intuitive 'gestures^ may be 
defined to control the operation of a virtual keyboard (column 6, lines 7 and 8) 
corresponding to means for associating gesture classes with individual typing keys. 

As for claim 16, Evans et al. teach the process loops back to select another 
word. This process continues until all the words have been placed in the data structure 
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(column 4, lines 62-64) corresponding to providing training data in words or sentences 
with certain timing data. 

Response to Amendment 

Applicant's arguments filed 8-18-2004 have been fully considered but they are 
not persuasive, applicant's representative argued that the prior art, Korth, does not 
teach a invisible virtual keyboard. Examiner disagrees with the applicant because the 
prior art, Korth teaches in the abstract "The activities are then interpreted as operations 
on a physically non-existent input device" corresponding to a invisible virtual keyboard. 
If one can touch an input device, it means that that input device is invisible or not 
physical. The applicant argued that the prior art, Evans et al., does not teach "the 
computer processes, those finger movements are transformed into text and classify the 
finger gestures into classes, and associate each of these classes with one of the keys 
of the keyboard". Examiner disagrees with the applicant because even if the prior art 
does not teach word for word the limitation claimed but the concept is inherently there. 
The prior art teaches whereby each key, or finger, would represent several different 
alphanumeric characters (column 2, lines 40 and 41 ) where the examiner is interpreted 
in Fig.1 as each key or finger represents different classes of gestures. And it says 
again further down in the prior art that the left-hand ring finger, for example, could 
represent "s", "v", and "x", the right-hand index finger could represent "j", "u," y\ "h" 
and "n", and so on for each finger (column 2, lines 43-46) where the left-hand ring 
finger represents a class of gestures and the right-hand index finger represents 
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another class of gestures. The applicant argued that the prior art does not teach "each 
word is identified on the basis of the probabilities assigned to the possible keys 
assigned to the gesture classes that make up that word" but examiner does not agree 
with the applicant because the prior art, Evans et al., teaches the word cat nnight 
become 228 if the letters c and a were both overloaded on the finger labeled key 2, 
and the letter t was on the finger 8 (column 4, lines 43-46) corresponding to "each word 
is identified on the basis of the probabilities assigned to the possible keys assigned to 
the gesture classes that make up that word". The claims have been amended but not 
enough to differentiate over the prior art. They need to be further amended to 
overcome the prior art of record. Therefore, the rejection is maintained as was rejected 
in the previous office action. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jean Lesperance whose telephone number is (703) 
308-6413. The examiner can normally be reached on from Monday to Friday between 
8:00AM and 4:30PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richard Hjerpe, can be reached on (703) 305-4709 . 

Any response to this action should be mailed to: 
Commissioner of Patents and Trademarks Washington, D.C. 20231 

or faxed to: 

(703) 872-9314 (for Technology Center 2600 only) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal drive, 
Arlington, VA, Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 
Jean Lesperance 
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